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* NOTICES * 

Japan Patent Office is not responsible for any 
damages caused by the use of this translation. 

1 .This document has been translated by computer. So the translation may not reflect the original precisely. 
2.**** shows the word which can not be translated. 
3. In the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application] this invention relates to the data processor which generates sound signals, such as musical sound and 

voice, in a multimedia device etc. 

[0002] 

[Description of the Prior Art] Conventionally, in synchronizing signals (it is called a "sound" below) in which a reproduction 
output is carried out by the loudspeaker etc. in a multimedia device, such as musical sound and voice, a character, a picture 
which are outputted on a display etc., etc. and making it output, the synchronizing signal for synchronizing these was 
generated independently, and it has taken the synchronization using this. 

[0003] Moreover, in order for what is necessary to be just to attach to the grade which human being can recognize in 
attaching words to musical sound, the words corresponding to the bottom of a note are written with kanji Japanese-syllabary 
mixture, katakana, the alphabet, etc. 
[0004] 

[Problem(s) to be Solved by the Invention] However, in the above-mentioned conventional example, since it is necessary to 
newly generate a synchronizing signal, there is a problem that the amount of data which a device treats increases. 
Furthermore, when you try to make it sing a song with a machine, the information concerning the length of reading or a note 
by the above-mentioned notation method is difficult for incorrectness and becoming complicated and eye an ambiguous 
hatchet and analysis making it sing correctly a machine. 

[0005] It sets it as the 1 st purpose that this invention offers the data processor which generates a synchronizing signal, 
without increasing the amount of data by having been made in view of the above-mentioned trouble, and assigning some 
sound data to a synchronizing signal. 

[0006] Furthermore, it sets it as the 2nd purpose to offer the data processor using the score means of displaying which 

performs analysis for making a song sing a machine easily and correctly. 

[0007] 

[Means for Solving the Problem] The data processor of this invention for attaining the 1 st purpose of the above is equipped 
with the following composition. Namely, the resetting means which reset the bit of the predetermined portion of the sound 
data which are the data processor which this is synchronized with sound signals, such as musical sound and voice, and 
processes image reconstruction etc., and consist of two or more bits, It has a superposition means to superimpose a 
synchronizing signal on the bit of the portion reset by the aforementioned resetting means in the aforementioned sound data, a 
voice reproduction means to reproduce the aforementioned sound data with which it was superimposed on the aforementioned 
synchronizing signal, and an extraction means to extract a synchronizing signal from the aforementioned sound data. 
[0008] Moreover, the data processor of this invention for attaining the 2nd purpose of the above is equipped with the 
following composition. Namely, a grant means to be the data processor which synthesizes voice based on the musical sound 
to which words were given, and to write the aforementioned musical sound by the note, and to make reading corresponding to 
the aforementioned note correspond to each note, and to give it, A 1 st extraction means to extract reading given in the 
aforementioned grant means as words information, A 2nd extraction means to judge a pitch based on the position of the 
aforementioned note, and to extract this as pitch information, The length which the note concerned has based on the 
configuration of the aforementioned note is judged, and it has a 3rd extraction means to extract this as length information, and 
a speech synthesis means to synthesize voice using the aforementioned words information, pitch information, and length 
information. 
[0009] 

[Function] By the above-mentioned composition, the bit of the predetermined portion of sound data is cleared and a 
synchronizing signal is superimposed to the bit portion. And while being superimposed on a synchronizing signal and 
carrying out voice reproduction of the sound data, the synchronizing signal on which it was superimposed is extracted. And 
processing of image reconstruction etc. is performed synchronizing with this synchronizing signal. 

[0010] Moreover, according to other composition, reading (words) corresponding to the musical sound and this which were 
written by the note is given corresponding to each note, each information on a pitch and length which the note corresponding 
to given reading and its reading expresses is extracted, and a song is made to sing a data processor by synthesizing voice. 
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[0011] 

[Example] With reference to an attached drawing, the suitable example of this invention is explained below. 
[00 1 2] <Example 1 > drawing 1 is a block diagram showing the outline composition of the voice / picture output unit of this 
example 1 . In drawing 1 , 1 is CPU and performs various kinds of control in this voice / picture output unit. 2 is ROM, the 
various control programs which CPU1 performs are stored, and the control program expressed with the flow chart of 
below-mentioned drawing 4 is also stored here. 3 is RAM and saves temporarily the data used in case CPU 1 performs 
various kinds of control. 4 is the input section and is constituted by the keyboard. A user inputs various data, control 
command, etc. from the input section 4. 5 is the external-memory section, for example, is constituted by the magnetic disk etc. 
Sound data 5a and image data 5b for a voice output are stored in the external-memory section 5. 

[00 1 3] 6 is the sound composition section, from sound data 5a read from the external-memory section 5, compounds a sound 
and outputs it to a loudspeaker 7 as a sound signal. 7 is a loudspeaker and carries out the reproduction output of the sound 
signal outputted from the sound composition section 6. 8 is the picture composition section, develops image data 5b read 
from the external-memory section 5 to an image data, and stores it in VRAM9. And although stored in VRAM9, the display 
output of the image data is carried out to a display 10. 

[0014] Next, above-mentioned sound data 5a is explained. In addition, the sound data of 16bitPCM(s) are used in this 
example. Drawing 2 is drawing showing the data structure of the conventional sound data, and drawing 3 is drawing showing 
the data structure of the sound data stored in the external-memory section 5 as sound data 5a. 

[0015] Drawing 2 expresses the data structure of the sound data of a basis, i.e., the sound data before synchronizing signal 
insertion, and drawing 3 expresses the data structure of the sound data after synchronizing signal insertion. Here, the bit 
position (synchronizing signal bit) which inserts a synchronizing signal is a portion which does not affect human being's 
acoustic sense most, and is a bit in which the slash portion of drawing 3 is equivalent to this in this example. And by 
extracting the synchronizing signal bit of this sound data, a synchronizing signal with an image data output is obtained, and 
the synchronization with a voice output and a picture output is taken. 

[0016] Next, the procedure which inserts a synchronizing signal in the sound data of above 16bitPCM(s) is explained 
according to the flow chart of drawing 4 . 

[00 1 7] First, original sound data are loaded in Step S2 1 . Original sound data are beforehand stored in the external -memory 
section 5. Next, in Step S22, the bit (synchronizing signal bit) for inserting a synchronizing signal is cleared. And in Step 
S23, the synchronization point which should be synchronized with data, such as a character and a picture, is chosen. Next, in 
Step S24, the synchronizing signal bit of the sound data in the synchronization point chosen at Step S23 is set. A 
synchronizing signal bit is set like "the time 3" and time "N-l" of sound data of drawing 3 as a result of this processing. And 
in Step S25, if check and it is [ whether there are any sound data which should otherwise be synchronized, and ], it will return 
to Step S23 and above-mentioned processing will be repeated. On the other hand, if there are no data which should 
synchronize, it will progress to Step S26, sound data with a synchronizing signal (sound data 5a) will be saved to the 
external-memory section 5, and this processing will be ended. 

[0018] A synchronizing signal can be generated without increasing the amount of data, since the bit of the one section of 
sound data is used for a synchronizing signal according to the voice / the picture output unit of this example, as explained 
above. Furthermore, since the bit of the portion used for a synchronizing signal uses the bit which does not have influence in 
human being's acoustic sense, even if it reproduces sound data with a synchronizing signal, it is almost uninfluential [ a bit ] 
on an acoustic sense. 

[0019] In the <example 2> example 2, the audio output device using the score means of displaying with words made possible 
[ making analysis required in order to make a song sing a machine correctly and easy ] is explained. In the audio output 
device of this example 2, in case words are attached to musical sound, it is unambiguous also in analysis equipment also from 
human being, and intelligible means of displaying is offered. 

[0020] Drawing 5 is a block diagram showing the outline composition of the audio output device of an example 2. In this 
drawing, 2 1 is CPU and performs various kinds of control in this audio output device. 22 is ROM and the various control 
programs which CPU21 performs are stored. The control program expressed with the flow chart of below-mentioned 
drawing 7 is also stored here. 23 is RAM, and CPU2 1 faces it performing control and it stores various data temporarily if 
needed. 24 is the input section and a user inputs various data and control command. 25 is the external-memory section, 
consists of magnetic disks etc. and stores the below-mentioned sound data 25a. 26 is the sound composition section, from 
sound data 25a read from the external-memory section 25, generates a sound signal and outputs it to a loudspeaker 28. A 
loudspeaker 27 carries out the reproduction output of the sound signal generated in the sound composition section 26. 28 is a 
display and performs various kinds of displays. 

[002 1 ] Drawing 6 is drawing showing the state where the note method of presentation by this example 2 was shown, and it 
was displayed on the display 28. A MIDI data input, a step input, etc. generate a score. Next, reading of the words according 
to the note in this score is given in hiragana to each note, and it displays into the note which corresponds as this is shown in 
drawing 6 . At this time, reading corresponding to the note of rice-flour dumplings, such as a whole note and a half note, is 
shown in the black, and the inverse video of the reading of the note of black shells, such as a quarter note and an eighth note, 
is carried out with a white character. By the method of presentation like an individual, a pitch, length, and reading are 
correctly written as one note. 

[0022] The inputted data are analyzed according to the flow chart shown in drawing 7 like drawing 6 . First, in Step S3 1 , one 
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note which should be analyzed is extracted and the character (words information) written in the note at Step S32 is read. 
Then, in Step S3 3, the position on 5 **** of the "ball" of the extracted note is judged, and a pitch is detected, and let this be 
pitch information. Furthermore, the configuration of a note is discriminated at Step S34, and the length information on the 
sound is acquired. And it judges whether processing was completed to all notes at Step S35, if it has not completed, it returns 
to Step S3 1, and above-mentioned processing is performed to the following note. On the other hand, this processing will be 
ended if all processings are completed at Step S3 5. 

[0023] The note shown in drawing 6 by the above-mentioned flow chart is stored in the external-memory section 25 as sound 
data as shown in drawing 8 . Drawing 8 is drawing showing one example of the data composition of sound data 25a. The 
information on reading of words is stored in words information 25b. Moreover, the information showing the pitch 
corresponding to each reading of words information is stored in pitch information 25c. Furthermore, the length of each note at 
the time of setting the length of a whole note to 1 is stored in 25d of length information. 

[0024] From the above sound data, the sound composition section 26 compounds a song using technique, such as rule 
composition. 

[0025] As explained above, according to the audio output device of this example 2, analysis for making a song sing a machine 
can be performed easily and correctly by writing reading of ** in the note of a score. 

[0026] In addition, it is possible to express clearly by the notation method in the above-mentioned example 2 also to the 
harmony from which a rhythm and words differ. An example of such harmony is shown in drawing 9 . And in the case of 
harmony, it generates separately with data composition as shows an upper part's sound data, and a lower part's sound data to 
drawing 8 . And two generated sound data are reproduced synchronously. 

[0027] In addition, even if it applies this invention to the system which consists of two or more devices, you may apply it to 
the equipment which consists of one device. Moreover, this invention cannot be overemphasized by that it can apply when 
attained by supplying a program to a system or equipment. 
[0028] 

[Effect of the Invention] It becomes possible to generate a synchronizing signal, without increasing the amount of data by 
assigning some sound data to a synchronizing signal according to the data processor of this invention, as explained above. 
[0029] Moreover, according to the data processor of other invention, the score means of displaying which performs analysis 
for making a song sing a machine easily and correctly is offered. 
[0030] 
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(57) Abstract: 

PURPOSE: To generate a synchronizing signal without 
increasing the data volume by assigning a part of sound 
data to the synchronizing signal 

CONSTITUTION: The data structure of sound data after 
synchronizing signal insertion is shown in the figure. The 
bit position where the synchronizing signal should be 
inserted is a part having the least influence upon the 
human hearing sense and corresponds to the part 
indicated by oblique lines in the figure. Synchronizing 
signal bits of sound data are extracted to obtain the 
synchronizing signal to a picture output, and the audio 
output and the picture output are synchronized with each 
other. When the synchronizing signal will be inserted to 
sound data, original sound data is first loaded. Next, bits 
for insertion of the synchronizing signal are cleared. 
Synchronous points to be synchronized with picture data 
are selected. Synchronizing signal bits of sound data are 
set to selected synchronous points. Synchronizing signal 
bits are set to a time 3 and a time N-1 of sound data by 
this processing. Thus, sound data with the synchronizing 
signal is saved in an external storage device. 
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